{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Pandas 描述性统计 - describe()\n",
        "\n",
        "本教程详细介绍如何使用 `describe()` 方法生成数据的描述性统计摘要。\n",
        "\n",
        "## 目录\n",
        "1. describe() 基础用法\n",
        "2. 数值型数据统计\n",
        "3. 分类型数据统计\n",
        "4. 自定义百分位数\n",
        "5. 按行和按列统计\n",
        "6. 高级应用\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 导入库\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "import warnings\n",
        "warnings.filterwarnings('ignore')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 1. describe() 基础用法\n",
        "\n",
        "### 方法说明\n",
        "\n",
        "`describe()` 方法用于生成描述性统计摘要，快速了解数据的分布情况。\n",
        "\n",
        "**语法:**\n",
        "```python\n",
        "df.describe(percentiles=None, include=None, exclude=None)\n",
        "```\n",
        "\n",
        "**主要参数:**\n",
        "- `percentiles`: 要包含的百分位数列表，默认 [0.25, 0.5, 0.75]\n",
        "- `include`: 要包含的数据类型，如 'all', 'object', 'number'\n",
        "- `exclude`: 要排除的数据类型\n",
        "\n",
        "**统计指标 (数值型数据):**\n",
        "- **count**: 非空值数量\n",
        "- **mean**: 平均值\n",
        "- **std**: 标准差\n",
        "- **min**: 最小值\n",
        "- **25%**: 第一四分位数 (Q1)\n",
        "- **50%**: 中位数 (Q2)\n",
        "- **75%**: 第三四分位数 (Q3)\n",
        "- **max**: 最大值\n",
        "\n",
        "**特点:**\n",
        "- ✅ 自动识别数值型列和分类型列\n",
        "- ✅ 返回统计摘要 DataFrame\n",
        "- ✅ 支持自定义百分位数\n",
        "- ✅ 可以指定包含或排除的列类型\n",
        "\n",
        "**适用场景:** 数据探索阶段，快速了解数据分布和异常值\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 示例1: 创建示例数据\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# 创建包含数值型和分类型数据的DataFrame\n",
        "np.random.seed(42)\n",
        "df = pd.DataFrame({\n",
        "    '年龄': np.random.randint(20, 60, 100),\n",
        "    '工资': np.random.normal(8000, 2000, 100),\n",
        "    '身高': np.random.normal(170, 10, 100),\n",
        "    '部门': np.random.choice(['技术', '销售', '人事', '财务'], 100),\n",
        "    '学历': np.random.choice(['本科', '硕士', '博士'], 100),\n",
        "    '绩效': np.random.choice(['A', 'B', 'C', 'D'], 100)\n",
        "})\n",
        "\n",
        "# 添加一些缺失值\n",
        "df.loc[10:15, '工资'] = np.nan\n",
        "df.loc[20:25, '年龄'] = np.nan\n",
        "\n",
        "print(\"原始数据 (前10行):\")\n",
        "print(df.head(10))\n",
        "print(\"\\n数据形状:\", df.shape)\n",
        "print(\"\\n数据类型:\")\n",
        "print(df.dtypes)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 示例2: 默认 describe() - 仅数值型列\n",
        "\n",
        "默认情况下，`describe()` 只统计数值型列（int64, float64）。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "print(\"默认 describe() 统计结果:\")\n",
        "print(df.describe())\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 2. 数值型数据统计详解\n",
        "\n",
        "### 示例3: 解释统计指标含义\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# 详细查看工资列的统计信息\n",
        "salary_stats = df['工资'].describe()\n",
        "print(\"工资列统计详情:\")\n",
        "print(salary_stats)\n",
        "print(\"\\n\" + \"=\"*50)\n",
        "print(\"统计指标解释:\")\n",
        "print(f\"count ({salary_stats['count']:.0f}): 非空值数量\")\n",
        "print(f\"mean ({salary_stats['mean']:.2f}): 平均值\")\n",
        "print(f\"std ({salary_stats['std']:.2f}): 标准差，衡量数据离散程度\")\n",
        "print(f\"min ({salary_stats['min']:.2f}): 最小值\")\n",
        "print(f\"25% ({salary_stats['25%']:.2f}): 第一四分位数，25%的数据小于此值\")\n",
        "print(f\"50% ({salary_stats['50%']:.2f}): 中位数，50%的数据小于此值\")\n",
        "print(f\"75% ({salary_stats['75%']:.2f}): 第三四分位数，75%的数据小于此值\")\n",
        "print(f\"max ({salary_stats['max']:.2f}): 最大值\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 示例4: 仅统计特定列\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# 只统计特定列\n",
        "print(\"仅统计年龄和工资列:\")\n",
        "print(df[['年龄', '工资']].describe())\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 3. 分类型数据统计\n",
        "\n",
        "### 示例5: 统计所有列（包括分类型）\n",
        "\n",
        "使用 `include='all'` 可以统计所有类型的列。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "print(\"统计所有列 (include='all'):\")\n",
        "print(df.describe(include='all'))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 示例6: 仅统计分类型列\n",
        "\n",
        "使用 `include=['object']` 只统计分类型（object类型）列。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "print(\"仅统计分类型列:\")\n",
        "print(df.describe(include=['object']))\n",
        "print(\"\\n分类型列统计指标说明:\")\n",
        "print(\"- count: 非空值数量\")\n",
        "print(\"- unique: 唯一值数量\")\n",
        "print(\"- top: 出现频率最高的值\")\n",
        "print(\"- freq: 最高频率值出现的次数\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 4. 自定义百分位数\n",
        "\n",
        "### 示例7: 指定自定义百分位数\n",
        "\n",
        "通过 `percentiles` 参数可以自定义要显示的百分位数。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# 自定义百分位数：10%, 50%, 90%, 95%, 99%\n",
        "print(\"自定义百分位数 [0.1, 0.5, 0.9, 0.95, 0.99]:\")\n",
        "print(df['工资'].describe(percentiles=[0.1, 0.5, 0.9, 0.95, 0.99]))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 示例8: 只显示中位数（50%百分位数）\n",
        "\n",
        "可以将 `percentiles` 设置为空列表，只显示基本的统计指标。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# 不显示百分位数，只显示基本统计\n",
        "print(\"不显示百分位数:\")\n",
        "print(df.describe(percentiles=[]))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 5. 按行和按列统计\n",
        "\n",
        "### 示例9: 按行统计（axis=1）\n",
        "\n",
        "默认按列统计（axis=0），可以改为按行统计。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# 创建纯数值型DataFrame用于行统计示例\n",
        "df_numeric = df[['年龄', '工资', '身高']].copy()\n",
        "print(\"原始数据 (前5行):\")\n",
        "print(df_numeric.head())\n",
        "print(\"\\n按行统计 (axis=1) - 统计每行的描述性信息:\")\n",
        "print(df_numeric.describe(axis=1).head())\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 6. 高级应用\n",
        "\n",
        "### 示例10: 排除特定列\n",
        "\n",
        "使用 `exclude` 参数排除不需要统计的列类型。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# 排除object类型的列\n",
        "print(\"排除object类型列:\")\n",
        "print(df.describe(exclude=['object']))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 示例11: 结合其他统计方法\n",
        "\n",
        "`describe()` 返回的是常用统计指标的快速概览，可以结合其他方法获取更详细的信息。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# 获取完整的统计信息组合\n",
        "stats_summary = pd.DataFrame({\n",
        "    'count': df['工资'].count(),\n",
        "    'mean': df['工资'].mean(),\n",
        "    'median': df['工资'].median(),\n",
        "    'std': df['工资'].std(),\n",
        "    'var': df['工资'].var(),\n",
        "    'min': df['工资'].min(),\n",
        "    'max': df['工资'].max(),\n",
        "    'skew': df['工资'].skew(),  # 偏度\n",
        "    'kurt': df['工资'].kurtosis()  # 峰度\n",
        "}, index=['工资'])\n",
        "\n",
        "print(\"详细的统计摘要:\")\n",
        "print(stats_summary.T)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 示例12: 检测异常值\n",
        "\n",
        "使用 `describe()` 的输出可以快速发现异常值（通过min和max）。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# 使用IQR方法检测异常值\n",
        "Q1 = df['工资'].quantile(0.25)\n",
        "Q3 = df['工资'].quantile(0.75)\n",
        "IQR = Q3 - Q1\n",
        "lower_bound = Q1 - 1.5 * IQR\n",
        "upper_bound = Q3 + 1.5 * IQR\n",
        "\n",
        "print(f\"工资列统计:\")\n",
        "print(df['工资'].describe())\n",
        "print(f\"\\n异常值检测 (IQR方法):\")\n",
        "print(f\"Q1 (25%): {Q1:.2f}\")\n",
        "print(f\"Q3 (75%): {Q3:.2f}\")\n",
        "print(f\"IQR: {IQR:.2f}\")\n",
        "print(f\"下界: {lower_bound:.2f}\")\n",
        "print(f\"上界: {upper_bound:.2f}\")\n",
        "\n",
        "outliers = df[(df['工资'] < lower_bound) | (df['工资'] > upper_bound)]\n",
        "print(f\"\\n异常值数量: {len(outliers)}\")\n",
        "if len(outliers) > 0:\n",
        "    print(\"异常值:\")\n",
        "    print(outliers[['姓名' if '姓名' in outliers.columns else '年龄', '工资']])\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 总结\n",
        "\n",
        "**describe() 方法的主要用途:**\n",
        "1. ✅ **快速数据概览**: 一行代码获得主要统计指标\n",
        "2. ✅ **异常值检测**: 通过min/max发现数据异常\n",
        "3. ✅ **数据分布了解**: 通过四分位数了解数据分布\n",
        "4. ✅ **数据质量检查**: 通过count检查缺失值情况\n",
        "5. ✅ **数据类型适配**: 自动适配数值型和分类型数据\n",
        "\n",
        "**最佳实践:**\n",
        "- 数据探索阶段首先使用 `describe()` 了解数据\n",
        "- 数值型数据默认统计即可\n",
        "- 需要完整统计时使用 `include='all'`\n",
        "- 结合可视化（如箱线图）更好地理解数据分布\n"
      ]
    }
  ],
  "metadata": {
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}
